-- Code review is a fundamental practice in software engineering, ensuring code quality, fostering collaboration, and reducing defects. While research has extensively examined var - ious aspects of this process, most studies assume that all code reviews follow a standardized evaluation workflow. However, our industrial partner, which uses Merge Requests (MRs) mechanism for code review, reports that this assumption does not always hold in practice. Many MRs serve alternative purposes beyond rigorous code evaluation. These MRs often bypass the standard review process, requiring minimal oversight. We refer to thes e cases as deviations, as they disrupt expected workflow patterns. For example, work - in - progress (WIP) MRs may be used as draft implementations without the intention of being review ed, MRs with huge changes are often created for code rebase, and library updates typically involve dependency version changes that require minimal or no review effort. We hypothesize that overlooking MR deviations can lead to biased analytics and reduced reliability of machine learning (ML) models used to explain the code review process. Our findings show that deviations occur in up to 37.02% of MRs across seven distinct categories. In addition, we develop a detection approach leveraging few - shot learning, achieving up to 91% accuracy in identifying these deviations. Furthermore, we examine the impact of removing MR deviations on ML models predicting code review completion time. Removing deviations significantly enhances model performance in 53.33% of cases, with improvements of up to 2.25 times. Our contributions include: (1) a clear definition and catego - rization of MR deviations, (2) a novel AI - based detection method leveraging few - shot learning, and (3) an empirical analysis of their exclusion impact on ML models explaining code review complet ion time. Our approach helps practitioners streamline review workflows, allocate reviewer effort more effectively, and ensure more reliable insights from MR analytics.

The advent of Large Language Models (LLMs) has revolutionized various domains of artificial intelligence, including the realm of software engineering. In this research, we evaluate the efficacy of pre-trained LLMs in replicating the tasks traditionally performed by developers in response to code review comments. We provide code contexts to five popular LLMs and obtain the suggested code-changes (patch sets) derived from real-world code-review comments. The performance of each model is meticulously assessed by comparing their generated patch sets against the historical data of human-generated patch-sets from the same repositories. This comparative analysis aims to determine the accuracy, relevance, and depth of the LLMs' feedback, thereby evaluating their readiness to support developers in responding to code-review comments. Novelty: This particular research area is still immature requiring a substantial amount of studies yet to be done. No prior research has compared the performance of existing Large Language Models (LLMs) in code-review comments. This in-progress study assesses current LLMs in code review and paves the way for future advancements in automated code quality assurance, reducing context-switching overhead due to interruptions from code change requests.

The automation of code review activities, a long-standing pursuit in software engineering, has been primarily addressed by numerous domain-specific pre-trained models. Despite their success, these models frequently demand extensive resources for pre-training from scratch. In contrast, Large Language Models (LLMs) provide an intriguing alternative, given their remarkable capabilities when supplemented with domain-specific knowledge. However, their potential for automating code review tasks remains largely unexplored. In response to this research gap, we present LLaMA-Reviewer, an innovative framework that leverages the capabilities of LLaMA, a popular LLM, in the realm of code review. Mindful of resource constraints, this framework employs parameter-efficient fine-tuning (PEFT) methods, delivering high performance while using less than 1% of trainable parameters. An extensive evaluation of LLaMA-Reviewer is conducted on two diverse, publicly available datasets. Notably, even with the smallest LLaMA base model consisting of 6.7B parameters and a limited number of tuning epochs, LLaMA-Reviewer equals the performance of existing code-review-focused models. The ablation experiments provide insights into the influence of various fine-tuning process components, including input representation, instruction tuning, and different PEFT methods. To foster continuous progress in this field, the code and all PEFT-weight plugins have been made open-source.

Sequence-to-sequence models have been used to transform erroneous programs into correct ones when trained with a large enough dataset. Some recent studies also demonstrated strong empirical evidence that code review could improve the program repair further. Large language models, trained with Natural Language (NL) and Programming Language (PL), can contain inherent knowledge of both. In this study, we investigate if this inherent knowledge of PL and NL can be utilized to improve automated program repair. We applied PLBART and CodeT5, two state-of-the-art language models that are pre-trained with both PL and NL, on two such natural language-based program repair datasets and found that the pre-trained language models fine-tuned with datasets containing both code review and subsequent code changes notably outperformed each of the previous models. With the advent of code generative models like Codex and GPT-3.5-Turbo, we also performed zero-shot and few-shots learning-based prompt engineering to assess their performance on these datasets. However, the practical application of using LLMs in the context of automated program repair is still a long way off based on our manual analysis of the generated repaired codes by the learning models.

Code review is a part of the software development process which involves testing the source code to identify bugs at an early stage. A code review process is typically conducted before merging with the codebase. An effective code review prevents bugs and errors from getting into your project by improving code quality at an early stage of the software development process. In this post, we'll explain what code review is and explore popular code review tools that help organizations with the code review process. The primary goal of the code review process is to assess any new code for bugs, errors, and quality standards set by the organization. The code review process should not just consist of one-sided feedback. Therefore, an intangible benefit of the code review process is the collective team's improved coding skills. If you would like to initiate a code review process in your organization, you should first decide who would review the code. If you belong to a small team, you may assign team leads to review all code.

In today's traditional code review processes, humans are a big part of how code is reviewed and ultimately committed to production. A person reads and comments on code to ensure it undergoes a peer review and is aligned with engineering processes and best practices. Code reviews started becoming an integral part of software delivery when code quality became a critical aspect to the business. Peer code reviews were initiated to enable additional "eyes" and input on code, improvements and possibly catch bugs and fixes early before it's shipped to production. However, like all aspects of software engineering, even the code review has greatly evolved, and today's code review needs to ensure a diversity of aspects before code is shipped to production, including: Each of these individually requires its own unique domain expertise.

Code review is an essential part to software development lifecycle since it aims at guaranteeing the quality of codes. Modern code review activities necessitate developers viewing, understanding and even running the programs to assess logic, functionality, latency, style and other factors. It turns out that developers have to spend far too much time reviewing the code of their peers. Accordingly, it is in significant demand to automate the code review process. In this research, we focus on utilizing pre-training techniques for the tasks in the code review scenario. We collect a large-scale dataset of real-world code changes and code reviews from open-source projects in nine of the most popular programming languages. To better understand code diffs and reviews, we propose CodeReviewer, a pre-trained model that utilizes four pre-training tasks tailored specifically for the code review scenario. To evaluate our model, we focus on three key tasks related to code review activities, including code change quality estimation, review comment generation and code refinement. Furthermore, we establish a high-quality benchmark dataset based on our collected data for these three tasks and conduct comprehensive experiments on it. The experimental results demonstrate that our model outperforms the previous state-of-the-art pre-training approaches in all tasks. Further analysis show that our proposed pre-training tasks and the multilingual pre-training dataset benefit the model on the understanding of code changes and reviews.

One of the things that always intrigues me is to look back at how a technology has evolved. Building programs to communicate with computers is something that has a particularly interesting history. Ada Lovelace is recognized as having written the first program to calculate Bernoulli numbers using Charles Babbage's Analytical Engine back in the mid-nineteenth century. Later in the following century came coding languages for lambda calculus and Turing machines making tape-based computations. Then assembly language, FORTRAN, LISP, COBOL, BASIC, C, and SQL came along with more general purpose and specialized programming languages leading up to the twenty-first century.